Apparatus and method for image processing and apparatus for coding

ABSTRACT

An image processing apparatus is provided that can define the change amount of a quantizing parameter by suitably reflecting the complexity of image data in the case, when the quantizing parameter is increased a predetermined unit amount, the coarseness the image data of motion picture is quantized becomes r times. An activity computing circuit generates an activity Nact j  serving as a complexity of the image data. A ΔQ computing circuit defines a corresponding relationship between activity Nact j  and change amount data ΔQ such that the change amount data ΔQ of quantizing parameter increases 1 as the activity Nact j  becomes 1.12 times, to thereby acquire change amount data ΔQ corresponding to the activity Nact j  generated by the activity computing circuit.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] This invention relates to an apparatus and method for imageprocessing and apparatus for coding that can define a quantizingparameter in a suitable way.

[0003] 2. Discussion of the Background

[0004] Recently, apparatuses conforming to the MPEG (Moving PictureExperts Group) scheme to handle image data are becoming more widespread.In such apparatuses compression is made by orthogonal transform such asdiscrete cosine transform and motion compensation through utilizingredundancy unique to image information for the purpose of efficienttransmission and storage of information, both in distributinginformation at the broadcast station and in receiving information at thegeneral household.

[0005] Particularly, MPEG2 (ISO/IEC13818-2) is defined as ageneral-purpose image coding scheme. This is the standard covering bothimages of interlaced scanning and progressive scanning as well as bothimages of standard resolution and high definition, now being broadlyused over a wide range of applications for professional and consumeruses.

[0006] The use of the MPEG2 compression scheme can realize highcompression efficiency and favorable image quality by assigning a codeamount (bit rate), for example, of 4-8 Mbps for a standard-resolutioninterlaced scanning image having 720×480 pixels or of 18-22 Mbps for ahigh-resolution interlaced scanning image having 1920×1088 pixels.

[0007] MPEG2 is mainly for high image-quality coding adapted forbroadcast, not suited for the lower code amount (bit rate) than MPEG1,namely the higher compressive coding scheme. The need for such a codingscheme is expected to expand in the future, due to the spread ofpersonal digital assistants. In order to cope with this, standardizationhas been completed on the MPEG4 coding scheme. Concerning the imagecoding scheme, the standard has been approved as an internationalstandard ISO/IEC14496-2, December 1998.

[0008] Furthermore, another standardization, called standard H.26L(ITU-T Q6/16 VCEG), has recently been pushed forward aiming at imagecoding initially for TV conferences. H.26L is known to realize highercoding efficiency despite requiring a much greater operation amount incoding and decoding as compared to the traditional coding schemes suchas MPEG2 and MPEG4. Meanwhile, Joint Model of Enhanced-Compression VideoCoding is now under standardization as part of MPEG4 activity, tointroduce functions not supported under the H.26L standard on the basisof the H.26L standards, and to thereby realize higher coding efficiency.

[0009] The coding apparatuses under the MPEG and H.26L standards realizeefficient coding by making most of the local pieces of information of animage.

[0010] An image has a nature that a complicated part of the image, evenif coded coarser in quantization than other parts, can be less visuallyrecognized of image deterioration.

[0011] For this reason, in the foregoing coding apparatus, the image isdivided into a plurality of parts to detect a complexity of the image oneach part. Based on the detection result, complicated parts of the imageare quantized coarsely, while the other parts are quantized finely,thereby reducing a data amount while suppressing the effect of imagedeterioration.

[0012] The information about image complexity is called activity.

[0013] In the foregoing coding apparatus, activity is computed on theimage data as a subject of quantization, to generate a quantizingparameter to regulate a quantizing scale on the basis of the activity.

[0014] In the meanwhile, the foregoing H.26L standard defines (PeriodicQuantization) to quantize the image data of motion picture by increasingcoarseness 1.12 times (12% increase) with increasing 1 in a quantizingparameter.

[0015] Accordingly, taking account of this, the present inventorsrecognized there is a necessity to generate the quantizing parameter onthe basis of the activity.

SUMMARY OF THE INVENTION

[0016] It is an object of the present invention to provide a novelapparatus and method for image processing and apparatus for coding thatcan define a change amount of a quantizing parameter by suitablyreflecting the complexity of image data in the case, when the quantizingparameter is increased a predetermined unit amount, the coarseness thatthe image data of motion picture is quantized becomes r times.

[0017] In order to achieve the foregoing object, an image processingapparatus of a first invention is an image processing apparatus forgenerating, when image data of a motion picture is made r times inquantization coarseness upon increasing a quantizing parameter apredetermined unit amount, change amount data representative of thechange amount of the quantizing parameter, the image processingapparatus comprising: index data generating means for generating indexdata serving as an index of complexity of the image data; and changeamount data acquiring means for defining a corresponding relationshipbetween the index data and the change amount data such that, when theindex data becomes r times, the change amount data is increased the unitamount, and acquiring the change amount data corresponding to the indexdata generated by the index data generating means.

[0018] The image processing apparatus of the first invention operates asfollows.

[0019] The index data generating means generates index data serving asan index of complexity of the image data.

[0020] Then, the change amount data acquiring means defines acorresponding relationship between the index data and the change amountdata such that, when the index data becomes r times, the change amountdata is increased the unit amount, and acquires the change amount datacorresponding to the index data generated by the index data generatingmeans.

[0021] An image processing method of a second invention is an imageprocessing method for generating, when image data of a motion picture ismade r times in quantization coarseness upon increasing a quantizingparameter a predetermined unit amount, change amount data representativeof the change amount of the quantizing parameter, the image processingmethod comprising: a first process of generating index data serving asan index of complexity of the image data; and a second process ofdefining a corresponding relationship between the index data and thechange amount data such that, when the index data becomes r times, thechange amount data is increased the unit amount, and of acquiring thechange amount data corresponding to the index data generated on thebasis of the definition in the first process.

[0022] A coding apparatus of a third invention comprises: index datagenerating means for generating index data serving as an index ofcomplexity of image data; change amount data acquiring means fordefining a corresponding relationship between the index data and thechange amount data such that, when the index data becomes r times, thechange amount data is increased the unit amount, and for acquiring thechange amount data corresponding to the index data generated on thebasis of the definition by the index data generating means; quantizingparameter generating means for generating the quantizing parameter onthe basis of reference data defined on the basis of a code amountassigned to the image data as a subject of coding and of the changeamount data acquired by the change amount data acquiring means; anorthogonal transform circuit for orthogonally transforming image data; aquantizing circuit for quantizing image data orthogonally transformed bythe orthogonal transform circuit; a quantizing control circuit forcontrolling quantization by the quantizing circuit such thatquantization coarseness is made r times as the quantizing parameter isincreased a predetermined unit amount, on the basis of the quantizingparameter generated by the quantizing parameter generating means; amotion predicting/compensating circuit for generating reference imagedata and a motion vector, on the basis of image data quantized by thequantizing circuit; and a coding circuit for coding image data quantizedby the quantizing circuit.

[0023] The coding apparatus of the third invention operates as follows.

[0024] The index data generating means generates index data to serve asan index of complexity of the image data.

[0025] Then, the change amount data acquiring means defines acorresponding relationship between the index data and the change amountdata such that, when the index data becomes r times, the change amountdata is increased the unit amount, and acquires the change amount datacorresponding to the index data generated on the basis of the definitionby the index data generating means.

[0026] Also, the orthogonal transform circuit orthogonally transformsimage data.

[0027] Then, the quantizing circuit quantizes the image dataorthogonally transformed by the orthogonal transform circuit.

[0028] At this time, the quantizing control circuit controlsquantization by the quantizing circuit such that quantization coarsenessis made r times as the quantizing parameter is increased a predeterminedunit amount, on the basis of the quantizing parameter generated by thequantizing parameter generating means.

[0029] Then, a motion predicting/compensating circuit generatesreference image data and a motion vector, on the basis of image dataquantized by the quantizing circuit.

[0030] Also, a coding circuit makes a coding on image data quantized bythe quantizing circuit.

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] A more complete appreciation of the present invention and many ofthe attendant advantages thereof will be readily obtained as the samebecomes better understood by reference to the following detaileddescription when considered in connection with the accompanyingdrawings, wherein:

[0032]FIG. 1 is a functional block diagram of a coding apparatusaccording to a background art to the present invention;

[0033]FIG. 2 is a functional block diagram of a decoding apparatusaccording to a background art to the present invention;

[0034]FIG. 3 is a figure showing a 4×4 zigzag scanning scheme in the JVTimage compression information;

[0035]FIG. 4 is a figure showing a coding scheme on the luminance signalcomponent of an intra 16×16 macro-block in the JVT coding scheme;

[0036]FIG. 5 is a figure showing a coding scheme on the chrominancesignal component of an intra 16×16 macro-block in the JVT coding scheme;

[0037]FIG. 6 is a figure showing the corresponding relationship betweena quantizing parameter QP_(y) on the luminance signal and a quantizingparameter QP_(c) on the chrominance signal, defined under the JVT codingscheme;

[0038]FIG. 7 is a figure for explaining a field/frame adaptive codingscheme in picture level, defined under the JVT coding scheme;

[0039]FIGS. 8A, 8B are figures for explaining a field/frame adaptivecoding scheme in macro-block level, defined under the JVT coding scheme;

[0040]FIG. 9 is a concept figure of a communications system according toa first embodiment of the present invention;

[0041]FIG. 10 is a functional block diagram of a coding apparatusaccording to the first embodiment of the present invention;

[0042]FIG. 11 is a figure showing the corresponding relationship betweenan input value Nact_(j) and an output value ΔQP, in a ΔQ computingcircuit; and

[0043]FIG. 12 is a figure showing a variable-sized motionpredicting/compensating block defined under the JVT coding scheme.

DETAILED DESCRIPTION OF THE INVENTION

[0044]FIG. 1 is a functional block diagram of a coding apparatus 500 ofa background art to the present invention.

[0045] In the coding apparatus 500 shown in FIG. 1, the input imagesignal is first converted into a digital signal in an A/D convertingcircuit 501. Then, depending upon a GOP (Group of Pictures) structure ofoutput image compression information, the frame image data is rearrangedin a screen-image rearranging circuit 502.

[0046] The image, to be subjected to intra-coding, in its entire offrame image data is input to an orthogonal transform circuit 504. In theorthogonal transform circuit 504, an orthogonal transform, such as adiscrete cosine transform or a Karhunen-Loeve transform, is carried out.

[0047] The orthogonal transform circuit 504 outputs a transformcoefficient to be quantization-processed in a quantizing circuit 505.

[0048] The quantizing circuit 505 outputs a quantized transformcoefficient to be input to a reversible coding circuit 506 where it issubjected to reversible coding such as variable-length coding orarithmetic coding. Then, it is stored to a buffer 507, and then outputas compressed image data.

[0049] The quantizing circuit 505 has a quantization rate controlled bythe rate control circuit 512. At the same time, the quantized transformcoefficient output from the quantizing circuit 505 is inverselyquantized in the inverse quantizing circuit 508, and subsequentlysubjected to an inverse orthogonal transform process in an inverseorthogonal transform circuit 509, to obtain reference frame image dataremoved of block distortion and decoded by the deblock filter 513. Thereference frame image data is stored in a frame memory 510.

[0050] Meanwhile, concerning the image to be subjected to inter-coding,the frame image data output from the screen-image rearranging circuit502 is input to a motion predicting/compensating circuit 511. At thesame time, reference frame image data is read from the frame memory 510,to generate a motion vector MV by the motion predicting/compensatingcircuit 511. Using the motion vector and the reference frame image data,predictive frame image data is generated. The predictive frame imagedata is output to an operating circuit 503. The operating circuit 503generates image data representative of a difference between the frameimage data from the screen-image rearranging circuit 502 and thepredictive frame image data from the motion predicting/compensatingcircuit 511. The image data is output to the orthogonal transformcircuit 504.

[0051] Meanwhile, the motion predicting/compensating circuit 511 outputsthe motion vector MV to the reversible coding circuit 506. In thereversible coding circuit 506, the motion vector is subjected to areversible coding process, such as variable-length coding or arithmeticcoding, and is inserted to a header of the image signal. The otherprocesses are similar to the processes of the image signal to besubjected to intra-coding.

[0052]FIG. 2 is a functional block diagram of a decoding circuit 499corresponding to the coding apparatus 500 shown in FIG. 1.

[0053] In the decoding circuit 499 shown in FIG. 2, input image data isstored in a buffer 613 and then output to a reversible decoding circuit614. In the reversible decoding circuit 614, the image data is subjectedto a process, such as variable-length decoding or arithmetic decoding,on the basis of a format of the frame image data. Simultaneously, in thecase that the relevant frame image data is an inter-coded one, themotion vector MV stored in the header of the frame image data is alsodecoded in the reversible decoding circuit 614. The motion vector MV isoutput to a motion predicting/compensating circuit 620.

[0054] The reversible decoding circuit 614 outputs a quantized transformcoefficient to be input to an inverse quantizing circuit 615 where it isinverse quantized. The inverse quantized transform coefficient issubjected, in the inverse orthogonal transform circuit 616, to aninverse orthogonal transform, such as an inverse discrete cosinetransform or an inverse Karhunen-Loeve transform, on the basis of apredetermined frame image data format. In the case that the relevantframe image data is an intra-coded one, the frame image data subjectedto inverse orthogonal transform process is removed of block distortionby a deblock filter 621 and then stored to a screen-image rearrangingbuffer 618, and is then output through D/A conversion process by a D/Aconverting circuit 619.

[0055] Meanwhile, in the case that the relevant frame is an inter-codedone, the motion predicting/compensating circuit 620 generates predictiveframe image data on the basis of the motion vector MV and the referenceframe data stored in the frame memory 621. The predictive frame imagedata, in the adder 617, is added by the frame image data output from theinverse orthogonal transform circuit 616. The subsequent processes aresimilar to the processes of the intra-coded frame image data.

[0056] Now, the inverse orthogonal transform process and inversequantizing process defined under the H.26L standard are described.

[0057] Under the H.26L standard, when carrying out a quantizing process,4×4 orthogonal transform coefficients are scanned over in the sequenceas shown in FIG. 3. In FIG. 3, “0”-“15” represent orthogonal transformcoefficients corresponding to totally sixteen pixel positions located ina 4×4 matrix form.

[0058] As shown in FIG. 4, in a 16×16 intra-macro-block 200, orthogonaltransform is carried out on each of the sixteen 4×4 orthogonal transformblocks 201 included in the relevant macro-block 200, thereaftercollecting only luminance DC coefficients having DC componentscorresponding to luminance position “0” thereof and again generating a4×4 block 202. This is subjected to orthogonal transform.

[0059] The 4×4 blocks 202 are scanned over in the sequence as explainedin FIG. 3.

[0060] Meanwhile, the luminance AC coefficients, as AC components shownby the remaining luminances “1”-“15” within the 4×4 orthogonal transformblock 201, are scanned zigzag in the sequence of starting at the secondposition (position at “1”) in a manner shown in FIG. 3.

[0061] Meanwhile, in the inverse quantizing process of the 16×16intra-macro-block under a chrominance signal component coding scheme,the chrominance DC coefficients within the 2×2 block 210 are firstscanned in a raster sequence, as shown in FIG. 5. Then, the chrominanceAC coefficients “1”-“15” remaining in the 4×4 chrominance blocks 211 arescanned zigzag in a sequence of starting at the second position(position at “1”) shown in FIG. 3.

[0062] A QP value, a quantizing parameter, is set by different values of0-51 in the number of 52.

[0063] A QP_(c) value, for use in chrominance, is defined withcorrespondence to a luminance QP_(Y) value as shown in FIG. 6.

[0064] The QP value is set to double the quantizing scale each time itincreases 6 (Periodic Quantization). Namely, the quantizing scaleincreases about 12% (becomes 1.12 times greater) as the QP valueincreases 1.

[0065] The coefficient R(m, i, j), to be used in the computationequation mentioned later, is computed by using a pseudo-code shown inthe following (1-1). $\begin{matrix}{R_{ij}^{(m)} = \left\{ \begin{matrix}V_{m0} & {{{for}\quad \left( {i,j} \right)} \in \left\{ {\left( {0,0} \right),\left( {0,2} \right),\left( {2,0} \right),\left( {2,2} \right)} \right\}} \\V_{m1} & {{{for}\quad \left( {i,j} \right)} \in \left\{ {\left( {1,1} \right),\left( {1,3} \right),\left( {3,1} \right),\left( {3,3} \right)} \right\}} \\V_{m2} & {{otherwise}\quad}\end{matrix} \right.} & \left( \text{1-1} \right)\end{matrix}$

[0066] The first and second subscripts on V in the above (1-1)respectively represent the row and column numbers of a matrix shown inthe following (1-2). $\begin{matrix}{V = \begin{bmatrix}10 & 16 & 13 \\11 & 18 & 14 \\13 & 20 & 16 \\14 & 23 & 18 \\16 & 25 & 20 \\18 & 29 & 23\end{bmatrix}} & \left( \text{1-2} \right)\end{matrix}$

[0067] After decoding the quantized DC coefficient of the 4×4 blockluminance component coded in the 16×16 intra-mode, an orthogonaltransform process is carried out in a procedure mathematicallyequivalent to the scheme as explained below. An inverse quantizingprocess is carried out after the orthogonal transform process.

[0068] The orthogonal transform process, on the DC coefficient of the4×4 block luminance component coded in the 16×16 intra-mode, is definedas in the following (1-3).

[0069] In the following (1-3), X_(QD) represents a matrix of theluminance DC coefficients after orthogonal transformation. The centermatrix on the right-hand represents a matrix of the luminance DCcoefficients before orthogonal transformation. $\begin{matrix}{X_{QD} = {{\begin{bmatrix}1 & 1 & 1 & 1 \\1 & 1 & {- 1} & {- 1} \\1 & {- 1} & {- 1} & 1 \\1 & {- 1} & 1 & {- 1}\end{bmatrix}\begin{bmatrix}y_{QD00} & y_{QD01} & y_{QD02} & y_{QD03} \\y_{QD10} & y_{QD11} & y_{QD12} & y_{QD13} \\y_{QD20} & y_{QD21} & y_{QD22} & y_{QD23} \\y_{QD30} & y_{QD31} & y_{QD32} & y_{QD33}\end{bmatrix}}{\quad\begin{bmatrix}1 & 1 & 1 & 1 \\1 & 1 & {- 1} & {- 1} \\1 & {- 1} & {- 1} & 1 \\1 & {- 1} & 1 & {- 1}\end{bmatrix}}}} & \left( \text{1-3} \right)\end{matrix}$

[0070] The image compression information conforming to thisspecification is not allowed to have an X_(QD)(i, j) value exceeding aninteger value within a range of −2¹⁵ to 2¹⁵ −1.

[0071] After the orthogonal transform process, inverse quantization iscarried out in the following procedure.

[0072] In the case that QP is a value of 12 or greater, inversequantization is made on the basis of the following (1-4).

[0073] Herein, DC_(ij) represents a DC coefficient inversely quantizedwhile F_(ij) represents a DC coefficient before inverse quantization.

DC _(ij) =[F _(ij) ·R ₀₀ ^((QP% 6))]<<(QP/6−2), i, j=0, . . . , 3  (1-4)

[0074] Meanwhile, in the case that QP is a value of 12 or smaller,inverse quantization is processed on the basis of the following (1-5).

DC _(ij) =[F _(ij) ·R ₀₀ ^((QP% 6))+2^(1-QP/6)]>>(2−QP/6), i, j=0, . . ., 3   (1-5)

[0075] The bit stream conforming to this specification is not allowed tohave a DC_(ij) value exceeding an integer value in the range of −2¹⁵ to2¹⁵−1.

[0076] After decoding the quantized DC coefficient in the 2×2 block ofchrominance component, an orthogonal transform process is carried out ina procedure mathematically equivalent to the following (1-6).$\begin{matrix}{X_{QD} = {{\begin{bmatrix}1 & 1 \\1 & {- 1}\end{bmatrix}\begin{bmatrix}i_{00} & i_{01} \\i_{10} & i_{11}\end{bmatrix}}\begin{bmatrix}1 & 1 \\1 & {- 1}\end{bmatrix}}} & \left( \text{1-6} \right)\end{matrix}$

[0077] The image compression information conforming to thisspecification is not allowed to have an X_(QD)(i, j) value exceeding aninteger value in the range of −2¹⁵ to 2¹⁵−1.

[0078] The inverse quantization after an orthogonal transform process onchrominance components is carried out in the following procedure.

[0079] In the case that QP is a value of 6 or the greater, inversequantization is made on the basis of the following (1-7).

DC _(ij) =[F _(ij) ·R ₀₀ ^((QP% 6))]<<(QP/6−1), i, j=0, . . . , 3  (1-7)

[0080] In the case that QP is a value smaller than 6, inversequantization is made on the basis of the following (1-8).

DC _(ij) =[F _(ij) ·R ₀₀ ^((QP% 6))]>>1, i, j=0, . . . , 3   (1-8)

[0081] The bit stream conforming to this specification is not allowed tohave a DC_(ij) value exceeding an integer value in the range of −2¹⁵ to2¹⁵−1.

[0082] The inverse quantizing process on 4×4 coefficients other than theforegoing is carried out on the basis of the following (1-9).

W _(ij) =[c _(ij) ·R _(ij) ^((QP% 6))]<<(QP/6), i, j=0, . . . , 3  (1-9)

[0083] Herein, R(m, i, j) is a coefficient defined in the foregoingequation (1-1).

[0084] The image compression information conforming to thisspecification is not allowed to have a W_(ij) value exceeding an integervalue in the range of −2¹⁵ to 2¹⁵−1.

[0085] After decoding the orthogonal transform coefficient matrix of 4×4block shown in the following (1-10), an orthogonal transform processtransforms the decoded orthogonal transform coefficient block into anoutput pixel value block in the mathematically equivalent procedureshown below. $\begin{matrix}{W = \begin{bmatrix}w_{00} & w_{01} & w_{02} & w_{03} \\w_{10} & w_{11} & w_{12} & w_{13} \\w_{20} & w_{21} & w_{22} & w_{23} \\w_{30} & w_{31} & w_{32} & w_{33}\end{bmatrix}} & \left( \text{1-10} \right)\end{matrix}$

[0086] Namely, one-dimensional orthogonal transform process is carriedout on the respective rows of decoded orthogonal coefficients.

[0087] Then, the similar one-dimensional transform process is made onthe determined matrix columns.

[0088] Herein, provided that the input orthogonal transform coefficientis W₀, W₁, W₂, W₃, an intermediate value Z₀, Z₁, Z₂, Z₃ is firstdetermined from the following (1-11), and furthermore a pixel value of adecoded image or a difference value thereof X₀, X₁, X₂, X₃ is determinedby the following (1-12).

Z ₀ =W ₀ +W ₂

Z ₁ =W ₀ −W ₂

Z ₂=(W ₁>>1)−W ₃

Z ₃ =W ₁+(W ₃>>1)   (1-11)

X ₀ =Z ₀ +Z ₃

X ₁ =Z ₁ +Z ₃

X ₂ =Z ₁ −Z ₃

X ×Z ₀ −Z ₃   (1-12)

[0089] The image compression information conforming to thisspecification is not allowed to have a value Z₀, Z₁, Z₂, Z₃, X₀, X₁, X₂,X₃ exceeding the integer value within the range of −2¹⁵ to 2¹⁵−1, in thefirst (horizontal) and second (vertical) orthogonal transform processes.Meanwhile, the bit stream conforming to this specification is notallowed to have a value X₀, X₁, X₂, X₃ exceeding the integer valuewithin the range of −2¹⁵ to 2¹⁵−1, in the second (vertical) orthogonaltransform process.

[0090] In both the horizontal and vertical directions, the following(1-13) block obtained by carrying out the orthogonal transform processis used, to compute a pixel value of decoded image or its differencevalue according to the following (1-14). $\begin{matrix}{X^{\prime} = \begin{bmatrix}x_{00}^{\prime} & x_{01}^{\prime} & x_{02}^{\prime} & x_{03}^{\prime} \\x_{10}^{\prime} & x_{11}^{\prime} & x_{12}^{\prime} & x_{13}^{\prime} \\x_{20}^{\prime} & x_{21}^{\prime} & x_{22}^{\prime} & x_{23}^{\prime} \\x_{30}^{\prime} & x_{31}^{\prime} & x_{32}^{\prime} & x_{33}^{\prime}\end{bmatrix}} & \left( \text{1-13} \right) \\{{X^{''}\left( {i,j} \right)} = {\left\lbrack {{X^{\prime}\left( {i,j} \right)} + 2^{5}} \right\rbrack 6}} & \left( \text{1-14} \right)\end{matrix}$

[0091] The final pixel value is computed by finally adding the decodedpredictive residual value X″(i, j) with a motion compensation predictivevalue or space predictive value P(i, j) as shown in the following (1-15)and making a clipping to include it within a range of 0 to 255.

S′ _(ij)=Clip1(P _(ij) +X″ _(ij))   (1-15)

[0092] In step 3 in a code-amount control scheme defined under TestModelof MPEG2 (“TestModel5”, ISO/IEC, JTC/SC29/WG11/N0400, 1993), a method isdefined to carry out adaptive quantization on the macro-block basis.Namely, by the method defined below, activity is changed on themacro-block basis, to quantize the reference quantizing scale code atgreater coarseness in a complicated pattern region where deteriorationis less visually noticeable.

[0093] Now, described are steps 1 to 3 in the rate control schemedefined under TestModel of MPEG2.

[0094] In step 1, the assigned bit amount to the pictures within a GOP(Group of Pictures) is distributed for the pictures not yet codedincluding the subject-of-assignment picture, on the basis of the bitamount R of assignment. This distribution is repeated in the order ofcoding pictures within the GOP. In such a case, the feature lies inmaking a code amount assignment to the pictures by the use of thefollowing first and second assumptions.

[0095] The first assumption is an assumption that the product of a meanquantizing scale code and a generation code amount used for coding eachpicture is constant on each picture type unless there is a change on thescreen.

[0096] Consequently, after coding the pictures, the parameter X_(I),X_(P), X_(B) (Global Complexity Measure) representative of screencomplexities is updated on each picture type according to the following(2-1).

[0097] With this parameter, it is possible to deduce the relationshipbetween a quantizing scale code and generation code amount for codingthe next picture. $\begin{matrix}\left. \begin{matrix}{X_{1} = {S_{1} \cdot Q_{I}}} \\{X_{P} = {S_{P} \cdot Q_{P}}} \\{X_{B} = {S_{B} \cdot Q_{B}}}\end{matrix} \right\} & \left( \text{2-1} \right)\end{matrix}$

[0098] Herein, S_(I), S_(P), S_(B) are generation code bits upon codingthe picture while Q_(I), Q_(P), Q_(B) are mean quantizing scale codes incoding the picture.

[0099] Meanwhile, the initial value is assumably a value shown by thefollowing (2-2), (2-3), (2-4) with the use of bit_rate [bits/sec] as atarget code amount.

X _(I)=160×bit_rate/115   (2-2)

X _(P)=60×bit_rate/115   (2-3)

X _(B)=42×bit_rate/115   (2-4)

[0100] The second assumption is to assume that, when the quantizingscale code ratio K_(P), K_(B) of P, B picture with reference to thequantizing scale code of I picture takes a value defined in thefollowing (2-5), the entire quality of image is to be optimized at alltimes.

K_(P)=1.0; K_(B)=1.4   (2-5)

[0101] Namely, the quantizing scale code of B picture is always 1.4times the quantizing scale code of I, P picture. This is based on theassumption that, in case B picture is quantized somewhat coarser ascompared to I, P picture to thereby add I, P picture with the codeamount saved on B picture, image quality is improved on I, P picture andfurther on B picture making a reference to that.

[0102] By the above two assumptions, the assignment code amount (T_(I),T_(P), T_(B)) to the pictures within the GOP is a value as representedby the following (2-6), (2-7), (2-8).

[0103] In the following (2-6), picture_rate represents the number ofpictures to be displayed per second (in the present sequence).$\begin{matrix}{T_{I} = {\max \left\{ {\frac{R}{1 + \frac{N_{P}X_{P}}{X_{I}X_{B}} + \frac{N_{B}K_{B}}{X_{I}K_{B}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & \left( \text{2-6} \right) \\{T_{P} = {\max \left\{ {\frac{R}{N_{P} + \frac{N_{B}K_{P}X_{B}}{K_{B}X_{P}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & \left( \text{2-7} \right) \\{T_{B} = {\max \left\{ {\frac{R}{N_{B} + \frac{N_{P}K_{B}X_{P}}{K_{P}X_{B}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} & \left( \text{2-8} \right)\end{matrix}$

[0104] Herein, N_(P), N_(B) is the number of P, B pictures not yet codedwithin the GOP.

[0105] Namely, concerning those of a picture type different from thepicture as a subject of assignment among the uncoded pictures within theGOP, deduction is made as to how many times in amount the generationcode of the subject-of-assignment pictures the relevant picturegenerates codes under the foregoing condition of image-qualityoptimization.

[0106] Then, it is determined to what number of subject-of-assignmentpictures the deduced generation code generated by the uncoded pictureentirety corresponds.

[0107] For example, N_(P)X_(P)/X_(I)K_(P) in the denominator second termof the first factor in the foregoing (2-6) represents how many sheets ofI pictures the N_(P) sheets of uncoded pictures within the GOP are to beconverted. This can be obtained by multiplying N_(P) by the ratioS_(P)/S_(I) of the generation code amount on P picture to the generationamount on I picture, to represent with X_(I), X_(P), K_(B) by using theforegoing (2-1), (2-5).

[0108] The bits on the subject-of-assignment picture are obtained bydividing the assignment bits R to the uncoded picture by that number ofsheets. However, the lower limit is set to the value in consideration ofcode amount required for the header or the like in a fixed fashion.

[0109] On the basis of the assignment code amount thus determined, eachtime the picture is coded according to the step 1, 2, the code amount Rto be assigned to the uncoded pictures within the GOP is updatedaccording to the following (2-9).

R=R−S _(I, P, B)   (2-9)

[0110] Meanwhile, when coding the first picture of the GOP, R is updatedaccording to the following (2-10). $\begin{matrix}{R = {\frac{{bit\_ rate} \times N}{picture\_ rate} - R}} & \left( \text{2-10} \right)\end{matrix}$

[0111] Herein, N is the number of pictures within the GOP. Meanwhile,the R initial value in the beginning of sequence is assumably rendered0.

[0112] Now, the step 2 is described.

[0113] In the step 2, in order to make the assignment bits (T_(I),T_(P), T_(B)) to the pictures determined in the step 1 agree with theactual code amount, a quantizing scale code is determined undermacro-block-based feedback control on the basis of the capacity of threekinds of virtual buffers independently set up for each picture type.

[0114] At first, prior to coding the j-th macro-block, the occupationamount on the virtual buffer is determined according to (2-11), (2-12),(2-13). $\begin{matrix}{d_{j}^{I} = {d_{0}^{I} + B_{j - 1} - \frac{T_{I} \times \left( {j - 1} \right)}{MBcnt}}} & \left( \text{2-11} \right) \\{d_{j}^{P} = {d_{0}^{P} + B_{j - 1} - \frac{T_{P} \times \left( {j - 1} \right)}{MBcnt}}} & \left( \text{2-12} \right) \\{d_{j}^{B} = {d_{0}^{B} + B_{j - 1} - \frac{T_{B} \times \left( {j - 1} \right)}{MBcnt}}} & \left( \text{2-13} \right)\end{matrix}$

[0115] d_(o) ^(I), d_(o) ^(P), d_(o) ^(B) is the initial occupationamount on each virtual buffer, B_(j) is the generation bits from apicture head to a J-th macro-block, and MBcnt is the number ofmacro-blocks within one picture.

[0116] The virtual buffer occupation amount (d_(MBcnt) ^(I), d_(MBcnt)^(P), d_(MBcnt) ^(B)) upon each end of picture coding is of the samepicture type, which is used as the initial value (d_(o) ^(I), d_(o)^(P), d_(o) ^(B)) of a virtual buffer occupation amount for the nextpicture.

[0117] Then, the quantizing scale code Q_(j) on the j-th macro-block iscomputed according to the following (2-14).

[0118] Herein, d_(j) is defined as (Equations 2-11 to 2-13) by the useof d_(j) ^(I), d_(j) ^(P), d_(j) ^(B). $\begin{matrix}{Q_{j} = \frac{d_{j} \times 31}{r}} & \left( {2\text{-}14} \right)\end{matrix}$

[0119] r is a parameter, called a reaction parameter, for controllingthe response speed over the feedback loop. This is given by thefollowing (2-15). $\begin{matrix}{r = {2 \times \frac{bit\_ rate}{picture\_ rate}}} & \text{(2-15)}\end{matrix}$

[0120] Note that the virtual-buffer initial value in the beginning ofsequence is given by the following (2-16). $\begin{matrix}{{d_{0}^{I} = {10 \times \frac{r}{31}}},{d_{0}^{P} = {K_{p}d_{0}^{I}}},{d_{0}^{B} = {K_{B}d_{0}^{I}}}} & \text{(2-16)}\end{matrix}$

[0121] Now, the step 3 is described.

[0122] Activity is provided by using the luminance signal pixel value ofthe original image instead of predictive errors and by using the pixelvalues of totally eight blocks of four 8×8 blocks in a frame DCT modeand four 8×8 blocks in a field DCT coding mode, according to thefollowing (2-17), (2-18), (2-19).

[0123] The following (2-18) shows var_sblk as a square sum ofdifferences between pixel data on each pixel and its mean value, havinga value increased as the image by the 8×8 block becomes complicated.$\begin{matrix}{{act}_{j} = {1 + {\min\limits_{{{sblk} = 1},8}\left( {{var}\quad {sblk}} \right)}}} & \text{(2-17)} \\{{{var}\quad {sblk}} = {\frac{1}{64}{\sum\limits_{K = 1}^{64}\left( {P_{k} - \overset{\_}{P}} \right)^{2}}}} & \text{(2-18)} \\{\overset{\_}{P} = {\frac{1}{64}{\sum\limits_{k = 1}^{64}P_{k}}}} & \text{(2-19)}\end{matrix}$

[0124] Herein, P_(K) is a pixel value within a luminance signal block ofthe original image. The reason for taking the minimum value (min) in theabove (2-17) is because of making quantization fine where there is aflat region even in a part within the 16×16 macro-block.

[0125] Furthermore, a normalization activity Nact_(j), whose value takesa range of 0.5-2, is determined according to the following (2-20).$\begin{matrix}{{Nact}_{j} = \frac{{2 \times {act}_{j}} + {avg\_ act}}{{act}_{j} + {2 \times {avg\_ act}}}} & \text{(2-20)}\end{matrix}$

[0126] which avg_act is the mean value of act_(j) in a picture codedimmediately before.

[0127] The quantizing scale code mquant_(j) taking account of visualcharacteristics is provided on the basis of a reference quantizing scalecode Q_(j), according to the following (2-21).

mquant _(j) =Q _(j) ×Nact _(j)   (2-21)

[0128] In the meanwhile, the JVT (Joint Video Team) image informationcoding apparatus may have input image information in aninterlaced-scanning format, similarly to the MPEG2 image informationcoding apparatus. The JVT image coding scheme defines a field/frameadaptive coding scheme in picture level and a field/frame adaptivecoding scheme in macro-block level, as described in the following.

[0129] Using FIG. 7, explanation is made on the field/frame codingscheme in picture level as defined under the JVT coding scheme.

[0130] Namely, it is possible, on each picture, to make a coding byselecting one higher in coding efficiency from frame coding and fieldcoding.

[0131] Using FIG. 8, explanation is made on the field/frame codingscheme in macro-block level as defined under the JVT coding scheme.

[0132] Namely, when implementing field/frame coding in macro-block levelunder the JVT coding scheme, scanning is made by taking two macro-blocksas a pair as shown in FIG. 8A. For each of the macro-block pairs, it ispossible to select whether to carry out field coding or frame coding, asshown in FIG. 8B.

[0133] In the meanwhile, the adaptive quantization as defined in theforegoing TestModel5 cannot be directly applied to the H.26L standardcoding scheme, because of the following two reasons.

[0134] The first reason is because Periodic Quantization is introducedunder the H.26L standard that quantization is made at twice coarsenesseach time quantizing parameter QP increases 6, i.e. quantizing scaleincreases about 12% (becomes 1.12 times) as QP increases 1.

[0135] Meanwhile, the second reason is because orthogonal transform ison an 8×8-block unit basis under the MPEG2 coding scheme whereasorthogonal transform is on a 4×4-block unit basis under the JVT codingscheme.

[0136] Now, explanation is made on an image processing apparatus of thepresent embodiment for solving the foregoing problem, and a method andcoding apparatus therefor.

[0137] First Embodiment

[0138]FIG. 9 is a concept diagram of a communications system 1 of thepresent embodiment.

[0139] As shown in FIG. 9, the communications system 1 has a codingapparatus 2 provided at the transmission end and a decoding apparatus499 provided at the reception end.

[0140] The coding apparatus 2 corresponds to the coding apparatus of theinvention.

[0141] The coding apparatus 2 and the decoding apparatus 499respectively perform a coding and a decoding, on the basis of theforegoing H.26L.

[0142] The decoding circuit 499 is the same as that mentioned beforeusing FIG. 2.

[0143] In the communications system 1, the coding apparatus 2 at thetransmission end generates compressed frame image data (bit stream) byan orthogonal transform, such as discrete cosine transform orKarhunen-Loeve transform, and motion compensation. The frame image data,after modulated, is transmitted through transmission mediums, such as asatellite broadcast wave, a cable TV network, a telephone line network,a cellular telephone line network, etc.

[0144] On the reception side, after demodulating a received imagesignal, the frame image data decompressed by inverse transform isgenerated and utilized to the orthogonal transform upon modulation andmotion compensation.

[0145] Note that the foregoing transmission medium may be a recordingmedium, such as an optical disk, a magnetic disk, a semiconductormemory, etc.

[0146] Incidentally, this embodiment is characterized in the method tocompute the change-amount data ΔQ of quantizing parameter in the codingapparatus 2.

[0147]FIG. 10 is an overall configuration diagram of the codingapparatus 2 shown in FIG. 9.

[0148] As shown in FIG. 10, the coding apparatus 2 has, for example, anA/D converting circuit 22, a screen-image rearranging circuit 23, anoperating circuit 24, an orthogonal transform circuit 25, a quantizingcircuit 26, a reversible coding circuit 27, a buffer 28, an inversequantizing circuit 29, an inverse orthogonal transform circuit 30, aframe memory 31, a rate control circuit 32, a motionpredicting/compensating circuit 36, a deblock filter 37, an activitycomputing circuit 40, and a ΔQ computing circuit 41.

[0149] The orthogonal transform circuit 25 corresponds to the orthogonaltransform circuit of the invention, the quantizing circuit 26 to thequantizing circuit of this embodiment, the reversible coding circuit 27to the coding circuit of the invention, the motionpredicting/compensating circuit 36 to the motion predicting/compensatingcircuit of the invention, the activity computing circuit 40 to theindex-data generator of the invention, the ΔQ computing circuit 41 to achange-amount data acquiring device of the invention, and the ratecontrol circuit 32 to a quantizing parameter generator of the invention.

[0150] The coding apparatus 2 carries out an orthogonal transform on a4×4-block unit basis according to the H.26L standard, and makes aquantization on the basis of the foregoing Periodic Quantization.

[0151] Now, explanation is made on the constituent elements of thecoding apparatus 2.

[0152] The A/D converting circuit 22 converts an input analog imagesignal constituted by a luminance signal Y and a chrominance signal Pb,Pr into a digital image signal, and outputs it to the screen-imagerearranging circuit 23.

[0153] The screen-image rearranging circuit 23 outputs the frame imagedata S23 that the frame image signals in the image signal input from theA/D converting circuit 22, after being rearranged in a coding orderaccording to a GOP (Group Of Pictures) structure including its picturetype I, P, B, to the operating circuit 24, to the motionpredicting/compensating circuit 36 and activity computing circuit 40.

[0154] When to inter-code the frame image data S23, the operatingcircuit 24 generates image data S24 representative of a differencebetween the frame image data S23 and the predictive frame image data S36a input from the motion predicting/compensating circuit 36, and outputsit to the orthogonal transform circuit 25.

[0155] Meanwhile, when to intra-code the frame image data S23, theoperating circuit 24 outputs the frame image data S23 as image data S24to the orthogonal transform circuit 25.

[0156] The orthogonal transform circuit 25 carries out an orthogonaltransform such as a discrete cosine transform or Karhunen-Loevetransform, on the image data S24, thereby generating image data (e.g.DCT coefficient signal) S25 and outputting it to the quantizing circuit26.

[0157] The orthogonal transform circuit 25 carries out an orthogonaltransform on a 4×4-block unit basis, according to the foregoing H.26Lstandard.

[0158] The quantizing circuit 26 quantizes the image data S25 into imagedata S26 with the use of a quantizing scale input from the rate controlcircuit 32, and outputs it to the reversible coding circuit 27 andinverse quantizing circuit 29.

[0159] The reversible coding circuit 27 stores to the buffer 28 theimage data 526 after being variable-length coded or arithmeticallycoded.

[0160] At this time, the reversible coding circuit 27 performs coding onthe motion vector MV input from the motion predicting/compensatingcircuit 36 or the difference thereof, and stores it to header data.

[0161] The image data stored in the buffer 28 is sent after beingmodulated or otherwise processed as needed.

[0162] The inverse quantizing circuit 29 generates the datainverse-quantized of the image data S26, and outputs it to the inverseorthogonal transform circuit 30, through deblock filter 37.

[0163] The inverse quantizing circuit 29 carries out the inversequantizing process on the basis of the H.26L standard, according to theforegoing equations (1-4), (1-5), (1-7), (1-8), (1-9).

[0164] The inverse orthogonal transform circuit 30 carries out aninverse orthogonal transform and the deblock filter 37 removes blockdistribution to form frame image data, and stores it to the frame memory31.

[0165] The inverse orthogonal transform circuit 30 carries out aninverse orthogonal transform on a 4×4-block unit basis as mentionedbefore, in compliance with the H.26L standard.

[0166] The rate control circuit 32 generates quantizing parameter QP onthe basis of the image data read from the buffer 28 and thechange-amount data ΔQP of quantizing parameter QP input from the ΔQcomputing circuit 41 and controls quantization by the quantizing circuit26 based on the quantizing scale corresponding to the quantizingparameter.

[0167] The rate control circuit 32 uses different values 0-52 as thequantizing parameter QP.

[0168] The QP_(c) value, for use in chrominance, is put withcorrespondence to the QP_(y) value of luminance, thus being defined asshown in FIG. 6.

[0169] Meanwhile, the rate control circuit 32 determines the quantizingscale such that the quantizing scale is doubled each time the quantizingparameter QP increases 6 (Periodic Quantization). Namely, the quantizingscale is increased about 12% (made 1.12 times) each time the quantizingparameter QP increases 1.

[0170] Similarly to the MPEG2 TestModel explained on the basis offoregoing (2-1) to (2-14), the rate control circuit 32 uses the codeamount (T_(I), T_(P), T_(B) in the foregoing (2-11), (2-12), (2-13))assigned to the image data as a subject of coding (picture) or the like,to generate a quantizing scale code Q_(j) on the basis of the foregoing(2-14) and take it as reference data OP_(ref) (reference data of theinvention).

[0171] In this case, the rate control circuit 32 acquires B_(j-1) by theforegoing (2-1), (2-12), (2-13), on the basis of the image data from thebuffer 28.

[0172] Then, the rate control circuit 32 adds the reference dataQP_(ref) and the change-amount data ΔQP together on the basis of thefollowing (3-1), to thereby generate a quantizing parameter QP.

QP=QP _(ref) +ΔQP   (3-1)

[0173] The motion predicting/compensating circuit 36 carries out amotion predicting/compensating process on the basis of the image dataS31 from the frame memory 31 and the image data from the screen-imagerearranging circuit 23, to generate a motion vector MV and referenceimage data S36 a.

[0174] The motion predicting/compensating circuit 36 outputs the motionvector MV to the reversible coding circuit 27 and the reference data S36a to the operating circuit 24.

[0175] The activity computing circuit 40, when the image data S23(original image picture) is a progressive scanning image, uses itsluminance signal pixel value, to compute var_sblk (distributed data ofthe invention) on each of the four 8×8 blocks (second block of theinvention) within the 16×16 macro-block (first block of the invention),on the basis of the following (3-2), (3-3).

[0176] Herein, var_sblk is a square sum of differences between pixeldata on each pixel and its mean value, having a greater value as the 8×8block image becomes more complicated. $\begin{matrix}{{{var}\quad {sblk}} = {\frac{1}{64}{\sum\limits_{K = 1}^{64}\left( {P_{k} - \overset{\_}{P}} \right)^{2}}}} & \text{(3-2)} \\{\overset{\_}{P} = {\frac{1}{64}{\sum\limits_{k = 1}^{64}P_{k}}}} & \text{(3-3)}\end{matrix}$

[0177] Then, the activity computing circuit 40 obtains act_(j) by theuse of a minimum value (min (var_sblk)) of var_sblk computed on the four8×8 blocks on the basis of the following (3-4). $\begin{matrix}{{act}_{j} = {1 + {\min\limits_{{{sblk} = 1},4}\left( {{var}\quad {sblk}} \right)}}} & \text{(3-4)}\end{matrix}$

[0178] The activity computing circuit 40 computes an activity Nact_(j)(index data of the invention) on the basis of the following (3-5).

[0179] The avg_act in the following (3-4) is a mean value of act_(j) inthe pictures coded immediately before.

[0180] Herein, the activity Nact_(j) is normalized to a value fallingwithin a range of 0.5-2. $\begin{matrix}{{Nact}_{j} = \frac{{2 \times {act}_{j}} + {avg\_ act}}{{act}_{j} + {2 \times {avg\_ act}}}} & \text{(3-5)}\end{matrix}$

[0181] Incidentally, there is a proposal to implement amacro-block-based field/frame adaptive coding process also under H.26Lsimilarly to that being implemented under MPEG2, in the document “MBadaptive field/frame coding for interlace sequences” (Wang et al,JVT-D108, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, ISO/IECJCT1/SC29/WG11 and ITU-T SG16 Q.6, Klagenfurt, Austria, July, 2002).However, in the case of making a coding process on this scheme, processis on each 8×8 block as a unit to compute an activity.

[0182] The ΔQ computing circuit 41 uses the activity Nact_(j) input fromthe activity computing circuit 40 to carry out, for example, anoperation shown in the following (3-6) and generate change-amount dataΔQP of quantization parameter QP.

ΔQP=└log_(1.12)Nact_(j)┘  (3-6)

[0183] Namely, the ΔQ computing circuit 41 defines the relationship ofactivity Nact_(j) and change-amount data ΔQP such that the change-amountdata ΔQP increases 1 (predetermined unit amount of the invention) asactivity Nact_(j) becomes 1.12 times (r times in the invention).

[0184] This can provide a definition that, in case the activity Nact_(j)representative of a complexity of the image as a subject of quantizationis doubled, the quantizing parameter QP increases 6 correspondingly tothereby double the quantizing scale.

[0185] Namely, it is possible to realize the Periodic Quantization asdefined under H.26L that the quantizing scale is increased about 12%(increased to 1.12 times) each time the quantizing parameter QPincreases 1.

[0186] Incidentally, the ΔQ computing circuit 41, when the activityNact_(j) lies between the minimum and maximum values shown in FIG. 11,may output change-amount data ΔQP at a value corresponding to that onthe basis of the table data 80 shown in FIG. 11, for example.

[0187] Now, explanation is made on the overall operation of the codingapparatus 2 shown in FIG. 10.

[0188] The input image signal, in the A/D converting circuit 22, isfirst converted into a digital signal. Then, the screen-imagerearranging circuit 23 makes a rearrangement of frame image data,depending upon a GOP structure of the image compression information tobe output.

[0189] Then, the activity computing circuit 40 generates an activityNact_(j) and outputs it to the ΔQ computing circuit 41.

[0190] Then, the ΔQ computing circuit 41 generates change-amount dataΔQP on the basis of the activity Nact_(j) such that the change amountdata ΔQP increases 1 as the activity Nact_(j) becomes 1.12 times, andoutputs it to the rate control circuit 32.

[0191] The rate control circuit 32 adds the reference data QP_(ref) andthe change amount data ΔQP together, to generate a quantizing parameterQP.

[0192] Meanwhile, concerning the frame image data to be intra-coded, theimage information in the frame image data entirety is input to theorthogonal transform circuit 25. In the orthogonal transform circuit 25,an orthogonal transform, such as a discrete cosine transform or aKarhunen-Loeve transform, is carried out.

[0193] The orthogonal transform circuit 25 outputs a transformcoefficient to be quantized in the quantizing circuit 26.

[0194] The quantizing circuit 26 makes a quantization by the quantizingscale defined based on the quantizing parameter QP under the control bythe rate control circuit 32.

[0195] The quantized transform coefficient, output from the quantizingcircuit 26, is input to the reversible coding circuit 27 where it issubjected to reversible coding, such as variable-length coding orarithmetic coding. Thereafter, it is stored to the buffer 28 and thenoutput as compressed image data.

[0196] At the same time, the quantized transform coefficient output fromthe quantizing circuit 26 is input to the inverse quantizing circuit 29,and is further subjected to an inverse orthogonal transform process inthe inverse orthogonal transform circuit 30 to generate decoded frameimage data. The frame image data is stored in the frame memory 31.

[0197] Meanwhile, concerning the image to be inter-coded, the frameimage data S23 is input to the motion predicting/compensating circuit36. Meanwhile, the frame image data S31 of reference image is read fromthe frame memory 31 and output to the motion predicting/compensatingcircuit 36.

[0198] In the motion predicting/compensating circuit 36, a motion vectorMV and predictive frame image data S36 a are generated by the use of thereference-image frame image data S31.

[0199] Then, in the operating circuit 24, image data S24 is generated asa difference signal between the frame image data from the screen-imagerearranging circuit 23 and the predictive frame image data S36 a fromthe motion predicting/compensating circuit 36. The image data S24 isoutput to the orthogonal transform circuit 25.

[0200] Then, in the reversible coding circuit 27, the motion vector MVis processed by a reversible coding, such as variable length coding orarithmetic coding, and inserted to the header of the image data. Theother processes are similar to the processes on the image data to beintra-processed.

[0201] As explained above, according to the coding apparatus 2, the ΔQcomputing circuit 41 defines the relationship between activity Nact_(j)and change-amount data ΔQP such that, in case the activity Nact_(j)becomes 1.12 times (r times in the invention), the change-amount dataΔQP increases 1 (predetermined unit amount of the invention), on thebasis of the foregoing (3-6) or the table data shown in FIG. 11. Thiscan properly reflect the value of activity Nact_(j) and realize thePeriodic Quantization defined under H.26L.

[0202] Meanwhile, according to the coding apparatus 2, althoughorthogonal transform is on the 4×4 block unit basis, the activitycomputing circuit 40 computes an activity Nact_(j) on the basis of theminimum value of var_sblk computed on the four 8×8 blocks within themacro-block. Consequently, the effect of adaptive quantization can beenhanced on the basis of the activity Nact_(j) appropriatelyrepresentative of a complexity distribution over the screen entirety.

[0203] Second Embodiment

[0204] This embodiment is similar to the first embodiment except in thatthe activity computing circuit 40 uses the 16×16 block as a computationunit for act_(j).

[0205] In this embodiment, the activity computing circuit 40 computesact_(j) on the basis of the following (3-7), (3-8), (3-9).$\begin{matrix}{{act}_{j} = {1 + {{var}\quad {sblk}}}} & \text{(3-7)} \\{{{var}\quad {sblk}} = {\frac{1}{256}{\sum\limits_{K = 1}^{256}\left( {P_{k} - \overset{\_}{P}} \right)^{2}}}} & \text{(3-8)} \\{\overset{\_}{P} = {\frac{1}{256}{\sum\limits_{k = 1}^{256}P_{k}}}} & \text{(3-9)}\end{matrix}$

[0206] In the case that one mask block is comparatively small in size ascompared to the image frame as in HDTV (High Definition Television),favorable image quality based on visual characteristics is available bycarrying out such adaptive quantization.

[0207] Incidentally, the 16×8 block or 8×16 block may be taken as acomputation unit for act_(j). Adaptive switching may be made betweenthese depending upon image local property.

[0208] In the meanwhile, it is possible to use a variable sized motionpredicting/compensating block under the H.26L standard, as shown in FIG.12.

[0209] In the inter-macro block, it can be considered that the motionpredicting/compensating block in the macro-block is used as a unit tocompute an activity.

[0210] In the case that the motion predicting/compensating block is in asub-partition mode, i.e. 8×8 or smaller, there is a possibility that itis impossible to sufficiently obtain an activity dispersion over theentire screen as per the foregoing. Consequently, 8×8 blocks may betaken as an activity computation unit.

[0211] Third Embodiment

[0212] Although the foregoing first embodiment exemplified the case thatthe input image data is of progressive scanning image data, the inputimage data in this embodiment is of an interlaced scanning image(interlace image). As mentioned before by using FIGS. 7 and 8,explanation is made on the case to carry out a field/frame adaptivecoding in a picture or micro-block level.

[0213] In the case of implementing a field/frame adaptive coding in apicture level as shown in FIG. 7 for example, when the relevant frame isfield-coded, the first and second fields are respectively consideredframes to thereby carry out an adaptive quantizing process similarly tothe case the input image data is progressive H scanned (the firstembodiment case), i.e. the processing by the activity computing circuit40, ΔQ computing circuit 41, rate control circuit 32, and quantizingcircuit 26 explained in the first embodiment.

[0214] Meanwhile, in the case of implementing a field/frame adaptivecoding in a macro-block level as shown in FIG. 8, the activity computingcircuit 40 computes act_(j) on one macro-block pair as explained in FIG.8A.

[0215] Namely, taking into account the cases of field-coding themacro-block pair and frame-coding the same, in the case of taking the8×8 block as a unit to compute slbk, the act_(j) of the relevantmacro-block pair is computed on totally 8 blocks due to frame-coding and8 blocks to be field-coded, i.e. totally 16 blocks, on the basis of thefollowing (3-10). $\begin{matrix}{{act}_{j} = {1 + {\min\limits_{{{sblk} = 1},16}\left( {{var}\quad {sblk}} \right)}}} & \text{(3-10)}\end{matrix}$

[0216] Meanwhile, in a case to take the 16×16 block as a computationunit for act_(j), the act_(j) of the relevant macro-block pair iscomputed on totally 2 blocks due to frame-coding and 2 blocks to befield-coded, i.e. totally 4 blocks, on the basis of the following(3-11). $\begin{matrix}{{act}_{j} = {1 + {\min\limits_{{{sblk} = 1},4}\left( {{var}\quad {sblk}} \right)}}} & \left( {3\text{-}11} \right)\end{matrix}$

[0217] Although the present invention was outlined on the exampleapplied to H.26L, the invention is not limited in application scope tothis, i.e. the present invention is applicable to an arbitrary imagecoding scheme using Periodic Quantization, 4×4 DCT.

[0218] Meanwhile, although the above embodiment exemplified the casethat the unit amount of the invention is 1 and r is 1.12, the unitamount and r may be other values.

[0219] According to the present invention, it is possible to provide animage processing apparatus capable of properly reflecting image datacomplexity and defining a change amount of quantizing parameter, and amethod therefor and a coding apparatus when image data of motion pictureis made r times in quantization coarseness upon increasing a quantizingparameter a predetermined unit amount.

[0220] Obviously, numerous modifications and variations of the presentinvention are possible in light of the above teachings. It is thereforeto be understood that within the scope of the appended claims, thepresent invention may be practiced otherwise than as specificallydescribed herein.

1. An image processing apparatus for generating, when image data ofmotion picture is made r times in quantization coarseness uponincreasing a quantizing parameter a predetermined unit amount, changeamount data representative of the change amount of the quantizingparameter, the image processing apparatus comprising: index datagenerating means for generating index data serving as an index ofcomplexity of the image data; and change amount data acquiring means fordefining a corresponding relationship between the index data and thechange amount data such that, when the index data becomes r times, thechange amount data is increased the unit amount, and for acquiring thechange amount data corresponding to the index data generated by theindex data generating means.
 2. An image processing apparatus accordingto claim 1, further comprising quantizing parameter generating means forgenerating the quantizing parameter based on reference data definedbased on a code amount assigned to the image data as a subject of codingand of the change amount data acquired by the change amount dataacquiring means.
 3. An image processing apparatus according to claim 1,wherein the index data generating means computes, based on a pluralityof second blocks as a unit defined within a first block of the imagedata, dispersion data representative of a dispersion of pixel datawithin the second block, and generates the index data by using a minimalone of the dispersion data among the dispersion data computed on theplurality of second blocks.
 4. An image processing apparatus accordingto claim 3, wherein the index data generating means computes thedispersion data by cumulating values depending upon a difference betweenpixel data within the second block and a mean value of all pixel datawithin the second block.
 5. An image processing apparatus according toclaim 3, wherein the index data generating means computes the dispersiondata based on the second block as a unit greater in size than a blockserving as a unit of making an orthogonal transform on the image data.6. An image processing apparatus according to claim 1, wherein, when theimage data is structured by a first field and a second field, the indexdata generating means generates the index data respectively on the firstfield and the second field, the change amount data acquiring meansacquiring the change amount data on the respective first and secondfields based on the index data generated by the index data generatingmeans.
 7. An image processing apparatus according to claim 3, whereinthe index data generating means computes the dispersion data on theplurality of second blocks defined within a plurality of the firstblocks when the image data is interlaced scanning image data.
 8. Animage processing apparatus according to claim 6, wherein the index datagenerating means computes the dispersion data on the plurality of secondblocks including the second block corresponding to field coding and thesecond block corresponding to frame coding.
 9. An image processingmethod for generating, when image data of motion picture is made r timesin quantization coarseness upon increasing a quantizing parameter apredetermined unit amount, change amount data representative of thechange amount of the quantizing parameter, the image processing methodcomprising: a first process of generating index data serving as an indexof complexity of the image data; and a second process of defining acorresponding relationship between the index data and the change amountdata such that, when the index data becomes r times, the change amountdata is increased the unit amount, and of acquiring the change amountdata corresponding to the index data generated in the first process. 10.An image processing method according to claim 9, further comprising athird process of generating the quantizing parameter based on referencedata defined based on a code amount assigned to the image data as asubject of coding and of the change amount data acquired in the secondprocess.
 11. A coding apparatus comprising: index data generating meansfor generating index data serving as an index of complexity of imagedata; change amount data acquiring means for defining a correspondingrelationship between the index data and the change amount data suchthat, when the index data becomes r times, the change amount data isincreased the unit amount, and for acquiring the change amount datacorresponding to the index data generated by the index data generatingmeans; quantizing parameter generating means for generating thequantizing parameter based on reference data defined based on a codeamount assigned to the image data as a subject of coding and of thechange amount data acquired by the change amount data acquiring means;an orthogonal transform circuit for orthogonally transforming imagedata; a quantizing circuit for quantizing image data orthogonallytransformed by the orthogonal transform circuit; a quantizing controlcircuit for controlling quantization by the quantizing circuit such thatquantization coarseness is made r times as the quantizing parameter isincreased a predetermined unit amount, based on the quantizing parametergenerated by the quantizing parameter generating means; a motionpredicting/compensating circuit for generating reference image data anda motion vector, based on image data quantized by the quantizingcircuit; and a coding circuit for coding image data quantized by thequantizing circuit.
 12. An image processing apparatus for generating,when image data of motion picture is made r times in quantizationcoarseness upon increasing a quantizing parameter a predetermined unitamount, change amount data representative of the change amount of thequantizing parameter, the image processing apparatus comprising: anactivity computing circuit to generate index data serving as an index ofcomplexity of the image data; and a ΔQ computing circuit to define acorresponding relationship between the index data and the change amountdata such that, when the index data becomes r times, the change amountdata is increased the unit amount, and to acquire the change amount datacorresponding to the index data generated by the activity computingcircuit.
 13. An image processing apparatus according to claim 12,further comprising a quantizing parameter generator to generate thequantizing parameter based on reference data defined based on a codeamount assigned to the image data as a subject of coding and of thechange amount data acquired by the ΔQ computing circuit.
 14. An imageprocessing apparatus according to claim 12, wherein the activitycomputing circuit computes, based on a plurality of second blocks as aunit defined within a first block of the image data, dispersion datarepresentative of a dispersion of pixel data within the second block,and generates the index data by using a minimal one of the dispersiondata among the dispersion data computed on the plurality of secondblocks.
 15. An image processing apparatus according to claim 14, whereinthe activity computing circuit computes the dispersion data bycumulating values depending upon a difference between pixel data withinthe second block and a mean value of all pixel data within the secondblock.
 16. An image processing apparatus according to claim 14, whereinthe activity computing circuit computes the dispersion data based on thesecond block as a unit greater in size than a block serving as a unit ofmaking an orthogonal transform on the image data.
 17. An imageprocessing apparatus according to claim 12, wherein, when the image datais structured by a first field and a second field, the activitycomputing circuit generates the index data respectively on the firstfield and the second field, the ΔQ computing circuit acquiring thechange amount data on the respective first and second fields based onthe index data generated by the activity computing circuit.
 18. An imageprocessing apparatus according to claim 14, wherein the activitycomputing circuit computes the dispersion data on the plurality ofsecond blocks defined within a plurality of the first blocks when theimage data is interlaced scanning image data.
 19. An image processingapparatus according to claim 17, wherein the activity computing circuitcomputes the dispersion data on the plurality of second blocks includingthe second block corresponding to field coding and the second blockcorresponding to frame coding.
 20. A coding apparatus comprising: anactivity computing circuit to generate index data serving as an index ofcomplexity of image data; a ΔQ computing circuit to define acorresponding relationship between the index data and the change amountdata such that, when the index data becomes r times, the change amountdata is increased the unit amount, and to acquire the change amount datacorresponding to the index data generated by the activity computingcircuit; a quantizing parameter generator to generate the quantizingparameter based on reference data defined based on a code amountassigned to the image data as a subject of coding and of the changeamount data acquired by the ΔQ computing circuit; an orthogonaltransform circuit for orthogonally transforming image data; a quantizingcircuit for quantizing image data orthogonally transformed by theorthogonal transform circuit; a quantizing control circuit forcontrolling quantization by the quantizing circuit such thatquantization coarseness is made r times as the quantizing parameter isincreased a predetermined unit amount, based on the quantizing parametergenerated by the quantizing parameter generator; a motionpredicting/compensating circuit for generating reference image data anda motion vector, based on image data quantized by the quantizingcircuit; and a coding circuit for coding image data quantized by thequantizing circuit.